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A Learner-Centered Design Method for Educational Technology 


Abstract 


Since 2003, Education Development Center, Inc. (EDC) has implemented projects that work directly with middle- 
school aged youth from underserved populations to develop digital resources aimed at encouraging young people 
to pursue science, technology, engineering, and mathematics (STEM) education and careers. It is through this 
work that EDC has developed and implemented a powerful, youth-centered, participatory research and design 
method for the design of educational technologies. This paper aims to share EDC’s learnings from developing 
and implementing this method including addressing issues of trust between youth and adult team members, 
appropriately acknowledging youth contributions, balancing the roles of mentors and adult design partners, and 
making dynamic curriculum adjustments based on participants learning styles and skill levels. 
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Since 2003, Education Development Center, Inc. (EDC) has implemented projects that work directly with middle- 
school aged youth from underserved populations to develop digital resources aimed at encouraging young people 
to pursue science, technology, engineering, and mathematics (STEM) education and careers. Through this work, 
EDC has developed and implemented a powerful, youth-centered, participatory research and design method for 
the design of educational technologies. This educational technology design method incorporates multiple elements 
and avenues for youth participation throughout the process of data collection. It consists of six phases, including 
elements based in research: conducting a literature review of existing topical research, gathering preliminary 
data on youth, gathering guiding data from youth, youth co-design team activities, product development, and 
evaluation. At the heart of this process, and the focus of this paper, are the youth co-design team activities. 
Each of the phases contributes a youth-centered point of view to the data collection process and, in turn, enables 
the products created in each project to be as youth-centered, youth-focused, and youth-informed as possible. 
Our underpinning assumption in this work is that youth involvement is essential to the process of designing 
products and activities specifically for middle school youth. To this end our educational technology design method 
incorporates multiple elements and avenues for youth participation throughout the process. Our resulting 
educational technology design method is thus an innovative approach to designing educational technologies with 
middle school youth for middle school youth. 


This paper shares EDC’s learnings from developing and implementing this participatory design approach in the 
development of educational technology. Three projects that yielded successful outcomes are presented as case 
studies to illustrate this method in action. These National Science Foundation-funded (NSF) projects include 
Girls Communicating Career Connections (GC3) and two projects that are part of the NSF’s National STEM Digital 
Library initiative (NSDL), namely The FunWorks and Middle School Portal 2 (MSP2): Math/Science Virtual Learning 
Experiences for Youth. 


Research Framework 


Our design methodology is rooted in a number of prior frameworks for participatory design with youth. The 
literature on collaborative and participatory design with children as design partners has been documented by 
researchers such as Druin, Soloway, Kafai, Scaife, Nesset and Large. 


In particular, the design theories that informed our work were Participatory Design (Carmel, Whitaker, & George, 
1993), Learner-Centered Design (Soloway, Guzdial, & Hay, 1994) and Cooperative Inquiry (Druin, 1999). Participatory 
Design is amethodology developed in Scandinavia used among adults (users and designers) in the users' workplace. 
It is based on several premises, including mutual reciprocal learning, design by doing, and cooperative prototyping. 
Low-tech prototyping is also an essential part of participatory design. This design methodology emphasizes 
the importance of taking into account the learner's needs (development of understanding, performance, and 
expertise) when designing a software. This theory emphasizes scaffolding as an important feature of any learning 
software. Cooperative Inquiry is another approach to creating new technologies for children and with children. 
This approach is based on other methods of participatory design and makes available a number of techniques that 
work well with children ranging from 7 to 10 years old, and sometimes even younger. 
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Our resulting educational technology design method is thus an innovative approach to designing educational 
technologies with middle school youth for middle school youth. The six phases of the design method are further 
described in the next section with a focus on each phase’s incorporation of youth participation. 


The Design Method 


As stated earlier, our design method implements the following six phases, each outlined in more detail below: (i) 
gathering preliminary date on youth (research/literature); (ii) gathering guiding data via surveys; (iii) gathering 
guiding data via focus groups; (iv) youth co-design team activities; (v) final product development; and (vi) pilot 
testing and evaluation. 


As stated earlier, the subsequent sections of this paper focus primarily on implementation of, and lessons learned, 
from the youth co-design team activities. 


Table 1. Six Phases of the Design Method 


1. Literature Review Established background and 
foundational information on the 
subject/content to be addressed 


Informs data collection needs for next 
steps 


2. Guiding Data: Surveys Provides broad youth perspective on 
the subject/content to be addressed 


3. Guidling Data: Focus Groups Offers rich data on youth perspectives | Focus group participants 
about specific topics and educational 
technologies 


4. Youth Co-Design Team Creates a design prototype, informed Design team participants 
by all available information and data 

5. Product Creation Translates the prototype into a Product reviewers 
finished, working product 

6. Pilot and Field Testing Verifies usability and allows for product | Field testers 
improvement 


Phase 1—Preliminary Data on Youth: Literature Review 


Literature regarding the issues or problems to be addressed by the project (e.g., STEM career information 
dissemination, middle school youth website preferences) is researched and synthesized. Additionally, any recent 
research utilizing youth participation and youth-centered approaches to data collection are also reviewed. Finally, 
most recent instruments for data collection and measurement with youth are reviewed with an eye towards the 
project at hand. This phase of our design method features little to no youth participation; rather, this phase focuses 
on gathering data about youth and youth participation. These data then help provide a context and framework to 
support the educators in how to engage youth in both the content and the design aspects of the project. 
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Phase 2—Guiding Data From Youth: Youth and Educator Online Surveys’ 


Online surveys, administered to both youth and their educators, provide current data and perspectives to supplement 
the content-specific information gathered through the literature review. While a review of the literature can offer 
a starting point for a project, information gleaned from a large, broad sample can inform and guide the project in 
its earliest stages. The online surveys, created using both original items and those from validated instruments, are 
easily accessible to both formal and informal educators in their learning spaces.* Educators have been identified 
as the ideal point of contact through which youth are recruited to participate in the survey. Typically, a separate 
educator survey is also developed to gain the perspective of those who directly work with the youth population 
of interest. Finally, to incent educators and their youth to complete the online surveys, gift cards are raffled to 
respondents. 


Phase 3—Guiding Data from Youth: Youth and Educator Focus Groups 


Shaped by information gathered from youth and educator online surveys, focus groups with youth from the target 
population provide contextual information that may be missing from survey data. Focus group protocols typically 
include items addressing key areas of the project; in addition, these protocols focus on the "how" as opposed to 
the "why" of the project at hand, providing depth of data where the surveys provide breadth. For instance, while 
survey items might identify youth website preferences, focus group data may yield information about how youth 
use these websites or how they came to identify these websites. Focus groups typically consist of 4—6 youth, and 
hands-on activities are used in conjunction with structured discussion in order to identify relevant data. As with 
the online surveys, this phase incorporates an educator component as well. 


Phase 4—The Heart of Participatory Design: The Youth Co-Design Team 


The youth co-design team is the heart of this educational technology design method. This phase is crucial to 
ensuring that learner-centered design principles are upheld in the creation of the end product. A group of youth, 
separate from those who participated in the survey and focus group activities, serve on a project-specific “youth 
design team,” functioning as experts and co-creators from start to finish of the product’s development. Design 
team members work with project staff to conceptualize, design, and test, with the intent of creating a prototype 
of a final product. The design team activities result in products that reflect the vision of these young designers as 
well as the needs and interests of their peers. 


1 Note that much of this work requires review by an Institutional Review Board (IRB). Drafting of proper consent forms for both youth and 
educator participants and tailoring these to each group in each phase can be an intensive, time-consuming process. Requirements include 
youth assent and educator consent forms, parental permissions, audio/video permissions, design team contracts for youth, and so forth. 
Permission for audio or visual recording should be incorporated into consent and assent forms to reduce the amount of paperwork for each 
participating family. For each of the cases discussed in this paper, IRB review was obtained prior to data collection, and both youth and 
educators were fully informed that data they were providing was collected as part of research to create project products. 


2 These include the ITEST Ocean Explorers Survey, the ITEST Eagle Vision Survey, and the OERL Evaluation of Impact on Teaching and 
Learning Survey. 
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Design teams convene in a series of classes or workshops that occur over the course of 8-12 weeks. Prior to 
convening a youth design team, facilitators and partner organizations engaged in essential logistics planning and 
curriculum development. Typically, youth from underserved populations have been recruited to be a part of 
youth design teams via partner organizations. Partner organizations also aid in identifying meeting space and in 
finalizing specific meeting dates and times. Familiar settings, people and organizations help youth develop a sense 
of “partnership” by bringing the project to them rather than requiring them to come to the project. 


Curriculum development consists of identifying key topics and skills that youth should gain by being a part of the 
design team and integrating project requirements into a plan that progressively teaches these topics and skills to 
participating youth. The curriculum integrates opportunities to develop communication and leadership skills and 
to learn project-specific content and skills. The curriculum is typically taught in the form of classes or workshops 
over the duration of the design team’s convening. 


Phase 5—Product Creation: The Final Product 


Upon completion of the prototype, all data gathered to this point (including the prototype itself) is reanalyzed and 
synthesized by project staff working with technology, media, and design specialists. Project staff are then tasked 
with creating a series of draft designs. Preliminary feedback is sought from project advisory board members, EDC 
and partner staff, and youth through either virtual or in-person presentations of the draft designs. This feedback is 
incorporated into the draft designs to create a final product. 


Phase 6—Evaluation: Pilot and Field Testing 


The final phase of the educational technology design method is to pilot and field-test the product’s feasibility of 
use in an educational setting, its youth appeal, and its effectiveness in attaining project goals. During this phase, 
additional groups of youth are asked to use the product and give feedback, answering questions about usability, 
appeal and effectiveness. Trained evaluators are typically brought in to provide both formative and summative 
assessments of the completed project and product. Any new information is used to improve the product(s) prior 
to its launch. 


Projects 


EDC’s first project to implement this design method, begun in 2003, was The FunWorks, a digital library of career 
exploration resources for youth ages 11-15. The site was developed by and for middle school aged youth and uses 
an array of experiential learning strategies to encourage diverse populations of youth to explore STEM careers. The 
FunWorks’ content and services also emphasize engaging currently underserved populations in STEM education 
and careers—females, minority populations, youth of low socioeconomic status (SES), and youth with disabilities. 
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In 2007, EDC began work on the Girls Communicating Career Connections (GC3). This project crafted a youth- 
produced, Web-based media series and companion educator materials on science and engineering careers, 
targeting middle school girls from underserved groups (minority populations, youth of low SES, and youth with 
disabilities). The videos encourage girls to recognize the science in their everyday lives, to understand its relevance 
to things most important to them now (e.g., sports, art, music), and to leverage that connection to spark interest 
in and knowledge of STEM careers. The GC3 video series is accompanied by educator materials for formal and 
informal educators that demonstrate how to effectively use and integrate the media series in various contexts. 
By doing so, it aims to allow educators to bring cutting-edge STEM career content into their classrooms and 
afterschool programs, thereby influencing the academic persistence and engagement of young girls exposed to 
the video series. 


The Middle School Portal 2 (MSP2): Virtual Learning Experiences for Youth project, begun in 2008, has developed 
youth-based math and science “virtual learning experiences” (VLEs). The youth VLEs are highly interactive, 
explorations for youth that encourage them to further explore and experience the math and/or science concepts 
associated with the EID into which the VLE is embedded. 


This paper will focus on the implementation of this process for the MSP2 project and its particular implementation 
of the youth co-design team and related activities. 
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CASE: The Middle School Portal 2 


Virtual Learning Experiences for Youth (SMARTR) 

The Middle School Portal 2 (MSP2): Math and Science Pathways is a project of The Ohio State University College of 
Education, the National Middle School Association (NMSA), and EDC. Funded by an NSF National STEM Distributed 
Learning (NSDL) grant in 2008, MSP2 supports middle school educators and youth with high-quality, standards- 
based resources and promotes collaboration and knowledge-sharing among its users. Educators use MSP2 to 
increase content knowledge in science, mathematics, technology and appropriate pedagogy for youth ages 10-15 
years. MSP2 also connects middle school youth to information on mathematics, science, and technology as well as 
health, safety, and career exploration. 


EDC led the development of Virtual Learning Experiences for Youth (SMARTR, http://smartr.edc.org), youth- 
based “virtual learning experiences” (VLEs) to be integrated into a pre-selected set of the MSP2’s Explore in 
Depth (EID) math and science publications for teachers. The VLEs are highly interactive explorations for youth, 
encouraging them to further explore and experience the math and/or science concepts associated with the topics 
in which they are embedded. Project staff utilized the educational technology design method in the development 
of VLEs as described below. 


Literature Review, Surveys and Focus Groups (Phase 1—3) 

Project staff conducted a literature review on topics including children as technology designers in order to inform 
the survey, focus group and, most importantly, youth design team activities. EDC staff then engaged in quantitative 
and qualitative research activities (in Spring 2009) to inform the design of the youth-centered Virtual Learning 
Experiences for the MSP2 portal. Middle school youth and educators participated in online surveys and focus 
groups. 


The youth survey and focus group primarily focused on youth preferences. The youth survey, with 440 responses, 
offered asnapshot into the types of websites middle school youth are interested in, their preferences for Internet use, 
and their abilities to consume and create online content. The youth focus group, conducted with five participants, 
offered a more detailed exploration of how youth navigate the Internet to find educational information as well as 
greater explanation of particular visual and layout preferences. 


The educator survey and focus group primarily focused on how educators utilize the Internet with their youth 
and how best to align VLE content with what educators teach in the classroom. The educator survey, with 617 
responses, offered insight into the technologies educators use with their youth, their perceptions of what engages 
their youth, and topics that are most frequently used to teach science and math concepts. The educator focus 
group, conducted with six participants, offered greater insight into the way in which educators identify and 
incorporate quality content into their teaching as well as suggestions for improvement of the current MSP2 site. 


Youth Co-Design Team (Phase 4) 
Partnering with a local community technology center, the project recruited nine middle school youth to participate 
in the design team. Over the course of eight weeks, the team met for two hours, twice a week at the center with 
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the goals of designing a STEM related website for middle school youth. The incentive for attendance and full 
participation was the receipt of an Apple iPod Touch upon completion of all design team work. Sessions often 
began with warm-up games to foster group bonding and creativity, leading into discussions about Web design and 
progressing on to session-specific agendas. These included three sets of activities. 


Identification of Relevant and Engaging Website Elements: This activity provided a foundation for subsequent 
activities. The design team explored what websites are attractive to middle schoolers and why. Exploration 
began with websites that participants used for fun and entertainment. A vocabulary for critiquing websites and 
website elements was introduced (including terms such as “menu,” “button,” “icon,” “font,” “link,” “navigation,” 
and “image”), and participants were asked to identify websites, complete worksheets about site elements, demo 
those sites for other participants, and lead group critiques of those sites. Design team members were often asked 
to work in groups during these activities, and the composition of groups was often changed in order to reinforce 
group cohesiveness. Often, these activities were performed on the center’s SMARTBoard, which allowed team 
members to do more interactive presentations. Design team participants repeated the same techniques for STEM 
content websites, both those sites that they had previously known and those provided by project staff. Activities 
were modified and extended to ensure that participants were comfortable using the vocabulary and could express 
themselves well using that terminology—and to gain as much insight into attractive Web design as possible. 


Creation of STEM Website Mock-ups: The second set of activities included individual and group website design, 
using both low- and high-tech tools to generate visual representations of participants’ designs; this mix of tools 
informs and builds upon one another and benefits middle school aged youth (Scaife et al., 1997). Paper mock-ups 
were initially used because they provide a simple and accessible means for youth to express their creative vision, 
as suggested by other design approaches (Carmel et al, 1993; Druin, 1999). Initially, youth created individual paper 
mock-ups of a general STEM website. This activity introduced participants to the design process and allowed them 
to share their uninfluenced ideas. After individual paper mock-ups were completed and critiqued, participants 
combined their ideas in groups of three, similar to the “mixing ideas” technique used by Guha et al. (2004). 
The “mixing ideas” technique, modified for middle schoolers, allowed participants to: come to a consensus of 
design elements that they like and dislike, exercise negotiating skills, refine design skills, and improve their overall 
contribution toward a final design. Once participants had their ideas on paper, they then used a simple, free 
website mock-up tool, Balsamiq, to refine their ideas, practice a different form of creative expression, and learn 
a new technology tool. Youth then created secondary pages about a single science or math topic using Balsamiq, 
which allowed them to learn to identify quality online content, continue their group work practice their technology 
skills. Finally, the entire design team combined their general STEM group pages into a single design. At this point 
during the sessions, team members had had adequate practice working in groups, using the vocabulary, and 
critiquing use of Web design elements to be able to complete this task successfully. 


Critique of STEM Sites and Mock-Ups: The third set of activities occurred in parallel with the design activities, as 
design and critique go hand in hand. Our “critiques” included writing down opinions, sharing opinions with the rest 
of the group during peer led-discussions, and discussing how design elements were intertwined with the creation 
of paper and Balsamiq mock-ups. Critiques always occurred as a group process moderated by project staff. In 
groups of three, participants engaged their fellow team members in a discussion about their likes or dislikes, 
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improvements, and perceived public opinion regarding a particular mock-up. Despite initial difficulty in getting 
youth to understand critiquing as a means of constructive criticism and idea improvement, participants became 
empowered to voice their opinions, practice use of technical vocabulary, apply concepts learned during phase one, 
hold a leadership role within the group, and engage one another in a constructive manner. 


The youth co-design team developed a consensus on their preferences and expectations for a website. In particular, 
participants voiced the need for “good” and attractive websites to include images, videos, blogs, chat areas, short 
links, large text in brightly colored fonts, and short descriptions of the site’s main topic/theme. On home pages, 
participants dislike large amounts of text, prefer short descriptions (in order to decide if they need to click/drill 
down further and read more about a particular topic), enjoy the ability to comment on a website’s material, chat 
with peers, and obtain homework help from an expert. They prefer to have the site’s content organized by grade 
level and subject. They also expect and want a search feature but ideally would like to have a search feature that 
restricts the results to no more than five sites. 


Other elements of design team activities: In addition, design team work also encompassed the activities below: 


¢ Team Building: Most design team members knew each other from school or from living in the same 
neighborhood; however, the team members were reticent to talk with one another about anything other 
than their day at school, and often there was no communication at all between the boys and the girls. They 
had never worked together on any type of project. Therefore, each session included an ice-breaker or team- 
building activity. Quick team activities unrelated to the project provided something productive and neutral 
to discuss. For example, one activity challenged them to work in two competing teams and make the longest 
chain possible out of provided scrap materials, and another activity required participants to ask questions 
about each other (e.g., “Find someone who has two siblings”). 


e Brainstorming and Group Decision Making: The brainstorming sessions were all structured in a similar manner. 
Beginning with a large question (e.g., “What images should go on the home page?”), design team members 
would jot down notes to themselves and then proceed to call out ideas while project staff members took notes 
on chart paper. Moving from brainstorming to consensus, team members were given a number of votes to 
distribute however they wished among their top choices. After two rounds of brainstorming, two rounds of 
narrowing, and much intermediate compromise, youth agreed upon the various elements of the sites basic 
design. 


e Field Assignments: Design team members were often assigned a task to complete and bring to the next 
meeting. Sometimes these assignments were tasks that could not be completed within the confines of the 
sessions, such as interviewing a friend about a particular math/science interest, researching and identifying a 
high-quality online source of math/science resources, etc. At other times, these assignments were to complete 
work begun during the sessions. 


Product Development (Phase 5) 

Upon creation of a prototype that received consensus from the youth co-design team, the low-tech prototype 
designs were delivered to professional Web designers at Missing Pixel, a New York-based multimedia/Web and 
technology design company. Prior to obtaining the final low-tech prototypes developed by the youth co-design 
team, a Missing Pixel staff member was invited to sit in on one of the youth co-design team sessions. The staff 
member actively participated in the session, asking questions and observing the youth’ design process. The youth 
also completed informal presentations of their mock-ups; the presence of the Missing Pixel staff person not only 
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provided Missing Pixel with some insight into the youth’ vision, but also solidified youth identities as product 
developers and experts. 


Missing Pixel staff members utilized the low-tech prototypes produced by youth in creating a series of Web design 
templates to use as a base for the VLE site, named SMARTR. Once a design template was chosen, a site was built 
and maintained by project staff using Drupal. 


A pilot test of the VLEs was developed by project staff and external evaluators and administered to youth and 
educators in Columbus, OH area. ASMARTR website review rubric was developed by the Evaluation Team from the 
Website Evaluation Form developed by Oklahoma State University (n.d.), based upon the work of Lynna Ausburn 
(2001). Using the rubric, the SMARTR website VLEs were rated on characteristics such as accuracy, accessibility, 
interactivity, eye-appeal, and appropriateness. The rubric consisted of 22 items rated on a scale using no (1), 
somewhat (2), and yes (3). The instrument was divided into three subscales. “Part 1: Quality and Source of 
Information” contained 6 items for a maximum score of 18. Part 1 items evaluated the accuracy, objectivity, and 
currency of the website. “Part 2: Technical Quality” consisted of 9 items for a maximum score of 27. “Part 3: 
Multimedia Components and Features” was a 7-item subscale with a maximum score of 21. Open-ended items 
were added to the rubric to collect qualitative data on target audience site usage. 
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SMARTR website reviews were then conducted by four middle school educators (two science educators and two 
mathematics educators) using the rubric. One science educator and one mathematics educator reviewed the 
SMARTR website in February 2011, while the other two educators reviewed the website in May 2011. SMARTR 
educator reviews were summarized. An Evaluation Team member administered a student questionnaire in two 
middle school classrooms, one Grade 6 Science classroom and one Grade 8 Mathematics classroom (for a total of 
43 students). Focus groups also were held in both classrooms to collect additional data on the SMARTR website. 
Results from the test are summarized in the next section. 


Outcomes 

SMARTR was soft launched in January 2011 with twelve science topics and ten math topics. Each topic includes 
content pulled from the NSDL’s MSP2 collection, highlighting youth-friendly games, activities, simulations, and 
career information. 


Data was also collected about the project’s impact on the youth design team participants. Data from a post- 
assessment administered to the co-design team demonstrates that participation in the design team did not 
change initial levels of interest in STEM careers, but there was a slight increase in technology interest, with more 
participants wanting to use technology in their future professions by the last session. When asked to identify 
science and math in everyday life at the beginning of the sessions, participants most often referred to money and 
nature; after the program they seemed to be able to identify other, unique ways in which STEM intersect with their 
personal everyday lives. 


With regard to computer knowledge, the Internet, and Web design, participants gained knowledge and skills such 
as the ability to describe the visual elements and basic structure of a Web page, explain how animation on a Web 
page can be a good or bad idea, explain how colors affect the look and feel of a webpage, draw/sketch a website 
on paper, use computer tools to sketch a website, and explain how to design a “good” Web page to a friend. In 
addition, most participants learned to rely on a website’s content and presentation as indicators of site quality in 
addition to triangulation (confirming with a teacher, parent, or peer that a site will provide correct information). 
This indicates that by the end of the sessions, participants had built a larger repertoire of criteria by which to judge 
site quality. 


When asked about the design team sessions themselves, the majority of participants enjoyed learning to make 
websites; this fulfilled their initial desire to learn these skills in a fun and “cool” 
were able to recognize that they came away from the design team sessions with greater technology and design 
knowledge and a more robust skill set than when they began. 


way. Almost all participants 


Results from Pilot Tests 

All four middle school educators who reviewed the SMARTR site found the information to be correct and error 
free, fair and unbiased, timely and up-to-date, and that the number of topics and depth of information were good 
for the target audience. Middle school education experts also found the information to be accurate and objective. 
They also indicated that the website contained information that was presented in an appropriate manner for the 
target audience and a search engine that worked easily and that website features were interactive and easily 
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accessed (no need for downloading additional software or plug-ins). All the educators agreed that students would 
enjoy the site if their teacher directed them to it for review or to enhance skills. 


Forty-three students responded to the student questionnaire. Twenty-one of these students were from a 
mathematics class and 22 were from a science class. The majority of students (91%) indicated that the topics were 
very easy or easy to find on the SMARTR website. They were asked to choose a math/science topic of interest and 
for the chosen topic, 5% of the students decided to experience a simulation on the SMARTR website, while 93% 
of the students decided to experience a game. After completing the simulations or games, 63% of the students 
indicated that the simulation or game helped them to understand the topic better, 79% indicated that they enjoyed 
the simulation or game, and 67% indicated that they had not used similar websites in their mathematics or science 
class. 


Further, eighty-four percent of the students indicated that the SMARTR website looked cooler than other 
mathematics/science websites and 72% of the students indicated that they would use the SMARTR website in 
the future to help with their mathematics or science skills. Focus groups were conducted as part of the pilot test 
consisted of a total of 5 females (2 mathematics, 3 science) and 4 males (2 mathematics, 2 science). Focus group 
participants indicated that they liked the appearance of the website. Two students commented on the layout of 
the homepage on which the approving comments focused on the main iPod graphic. Comments regarding the 
appearance of the webpage included that it looked “cool,” it had “colorful games,” and it made them “want to 
do stuff.” All respondents agreed that the site would be helpful for learning science or mathematics. One student 
explained why she thought it was helpful for learning science by stating, “you get to interact with it, not just 
learning out [of] the book.” The majority of students indicated that they would use the site on their own. 


Youth Co-Design Teams—Lessons Learned 


Over the course of these three projects, staff learned a number of important lessons about planning and executing 
a successful participatory youth design process, particularly with respect to the implementation of the youth co- 
design team. 


Careful Session Scheduling. Design team work often competed with sports schedules during the academic year 
and with and summer camp/summer vacation schedules when school is out. Also because design team members 
often attend different schools, school dismissal times vary as does the time required to travel from school to the 
design team site. Lastly, depending on where design team meetings are held, several other programs may share 
space with the design team. Since design team participants are compensated based on their attendance and 
participation, they are incented to negotiate the most viable days/times for session scheduling. 


Facilitate Effective Group Work. Young people who regularly socialize together because they are from the same 
neighborhood or attend the same school do not necessarily know how to work on a project together. Team- 
building and ice-breaker activities are essential to create design team cohesion. The design team members are 
able to identify constructive team skills during debriefing of team-building activities. 
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Communication Beyond Sessions. Outside of the design team sessions, it can be difficult to find viable and effective 
means of communication with design team members. Even when participants have their own email accounts, or 
are provided accounts by the project, barriers to effective e-mail communication include limited access to the 
Internet, lack of time or opportunity to check e-mail even when the Internet is available (such as during school), 
and in some cases relative newness to using e-mail as a means of communication. Often, contact via phone is a 
more reliable way of reaching team members. 


Memorandums of Understanding. To address scheduling and other issues described above, it is extremely 
beneficial to draft a Memorandum of Understanding between the project and the design team meeting host site 
specifying when the design team will use the allocated space, what equipment will be used, assigning responsibility 
to site staff for the safe transport of students to and from the meetings, and detailing how and when students will 
be compensated. 


Design Partner vs. Mentor. Project staff members who facilitate the design team meetings must act more as 
design partners and less as mentors. The reality of implementation, however, requires staff members to constantly 
“switch hats,” and all staff members invariably have both design partner and mentor moments. The activity plan/ 
curriculum for design team meetings should heavily emphasize having youth participants design by themselves. 
Staff can act as design partners through the brainstorming process and throughout the project as constant agents 
of support. Participants often saw project staff as mentors rather than design partners, and greater awareness of 
the differences in these roles is encouraged for future design teams. 


Trust and Incentives. Working with middle school youth as design partners requires an open mind, willingness to 
trust their ideas and contributions, and availability to become their peer whenever possible. The establishment of 
trust needs must be mutual between youth and project staff or youth will not likely to be as invested or engaged 
in the design process. 


Development of, and Revisions to, the Activity Plan. Through the course of each project’s design team meetings, 
the activity plan is constantly revised to accommodate the learning challenges and interests of the group. This 
iterative process is often the best approach to ensuring that activities meet the needs of participants and yet 
challenge them sufficiently. An overarching activity plan that is flexible enough for changes, depending on where 
the learners are on their learning trajectory, is highly recommended. 


The Impact of Importance. Participatory Design research highlights that success occurs only when participants 
believe their ideas are important (Ackoff, 1974). During one of the sessions for the MSP2 project, a professional 
Web designer observed participants activities. This professional provided suggestions, commented on designs, 
gave additional support, and mentioned how much she was in need of designers with their diverse background 
and age. This session provided a turning point for many participants; there was a noticeable change in trust and 
engagement among the design team participants as we moved forward. The fact that this professional had not 
established a day to day relationship with participants made her appearance novel and influential. 


Finally, as part of this work, we have developed a rubric/checklist that summarizes the intricacies of the six-phase 
design process and is included as an appendix to this paper. 
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Conclusion 


The experience gained via the MSP2/SMARTR and our prior projects, demonstrates not only what it takes to 
successfully employ a youth participatory design approach, but also that the most appealing and engaging activities 
for youth are effective and accurate data collection and product creation tools. In addition, using media as leverage 
in STEM engagement of youth can be highly effective. In terms of our design team participants, it is clear that 
providing opportunities for these youth to learn more about STEM and STEM topics and careers was as important 
as allowing them to see themselves in new roles (e.g., researcher, Web designer, etc). Their participation in the 
projects demonstrated to them that their thoughts and ideas matter, and that their personal stories, ideas and 
questions matter—and together these are crucial steps to imagining a future for themselves in a STEM field. 


This work builds upon more than a decade of research which has focused on designing technology for children 
and on how to involve children as participants in the design process. As stated earlier the work of researchers 
such as Druin, Kafai, Bilal and others illustrate different approaches of this process. Our experience has shown us 
that youth can be highly effective, creative and honest as design partners in educational technology development 
efforts. Children today grow up in a technology-saturated world and are often immersed in the world of interactive 
technologies even prior to starting school. Their experience with technologies, as digital natives, impacts both 
their understanding of technology, and their expectations of it, in fascinating ways. Still, designing technology tools 
with students presents some complex issues as outlined in this paper, and is a process that needs to be adequately 
scaffolded. And while it is obvious that the transition from a multimedia consumer to a multimedia producer is 
not an easy one, the potential benefits make it a worthwhile learning and development experience for all those 
involved. 
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Appendix—Design Rubric 


Phase 1—Preliminary Data on Youth: Literature Review 
Identification of relevant topic literature 
Identification of relevant methodology 
Identification of relevant instruments and tools 
Synthesis of gathered data 
Phase 2—Guiding Data From Youth: Youth and Educator Online Surveys 
Use of Phase 1 findings to construct preliminary surveys 
Augmentation of preliminary surveys with additional items based on project demands 
Identification of partners for survey dissemination 
Identification of incentives for survey participants 
Internal Review Board (IRB) review 
Creation of online surveys from finalized and approved instruments 
Dissemination of online surveys through partners 
Data collection 
Data analysis 
Phase 3—Guiding Data From Youth: Youth and Educator Focus Groups 
Use of Phase 1 and Phase 2 findings to construct preliminary focus group guides or protocols 
Augmentation of preliminary guides or protocols with additional items based on project demands 
Identification of partner sites for focus group completion 
Identification of incentives for focus group participants (in conjunction with partner sites) 
Internal Review Board (IRB) review 
Finalization of approved instruments 
Phase 4—The Heart of Participatory Design: The Youth Co-Design Team 
Use of Phases 1-3 findings to construct preliminary “curriculum” 
Identification of partner site(s) for youth co-design team sessions 
Scheduling and logistics with partner site(s), including incentive identification for co-design team participants 
Augmentation of preliminary curriculum with additional items based on project demands and partner site 
requirements 
Internal Review Board (IRB) review 
Finalization of approved curriculum 
Recruitment of youth co-design team via partner site(s) 
Youth design team sessions and data collection commence 
Youth co-design team creates preliminary prototype drafts of final product 


Data analysis 
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Phase 5—Product Creation: The Final Product 
Use of Phases 1-4 findings to inform drafts of a professional prototype 
Prototype drafts are narrowed and redrafted 
Final professional prototype is created 

Phase 6—Evaluation: Pilot and Field Testing 
Creation of evaluation tools 
Identification of partners for pilot testing and evaluation 
Identification of incentives for evaluation participants 
Internal Review Board (IRB) review 
Scheduling and logistics with partner site(s) 
Recruitment of youth and/or educators via partner site(s) 
Data collection 
Data analysis 
Refinement of prototype using evaluation findings 
Finalization of product 


